This invention relates generally to the representation of electronic information, and more specifically to representing data in a scripting tool.
The above-referenced, commonly assigned and co-pending U.S. application entitled USER DEVICE-INDEPENDENT TRANSACTION MODEL (the xe2x80x9cUDT Inventionxe2x80x9d) discloses a model for delivering generic information from a central system to connected users whose terminal devices communicate in varying formats and/or media. The UDT Invention teaches the use of Transaction Templates (TTs) and User Device Templates (UDTs) which script and condition device-independent information exchanges between the system and users to be compatible with the different format and media requirements of said users"" various terminal devices. In the UDT Invention, TTs enable an exchange of information between the system and the user in the information""s most basic, generic format. UDTs associated with each TT then condition the information exchanged in the TT to be format- and media-compatible with the device through which the user is communicating with the system. The same information exchange may thus be enabled in a variety of environments according to the formats and media supported by various users"" terminal devices.
A constraint placed upon the effectiveness of UDTs, however, is the available bandwidth in which to deliver information in the scripted formats and/or media. This is particularly applicable when the information is to be delivered in several media, the display of which to the user is to appear simultaneous and/or synchronized. It will be appreciated that, according to the UDT Invention, a UDT may be designed wherein an information transaction is scripted to appear to the user as a sophisticated multimedia experience, including sound, video, graphics and text, assuming that all of these media are supported by the user""s multimedia PC. If the available, bandwidth in the interconnecting link between system and user is insufficient to deliver all this information according to the time constraints of the script, however, this multimedia UDT will operate at reduced effectiveness.
One solution to this problem of insufficient bandwidth is for the user simply to sit and wait for a sufficient amount of information to have been delivered to begin the script with confidence that it can be completed without running out of information mid-script.
Another solution is to redesign either (or both) the sequence or the content of the script, so that the script""s delivery is either slower or less data-rich. For example, a display of merely ornamental graphics could be delayed or removed to make room for the earlier delivery of more strategic information. Alternatively, a media object may be degraded (e.g. sampling rate reduced) to make the object less data-rich.
A yet further solution is to preload information in advance of its being needed by the script. This solution may be enabled using excess bandwidth in early parts of a particular script. This may be done on an individual script basis, alternatively, when the system knows specifically that a later script will be played, information for that later script may be preloaded on the user""s terminal device using excess bandwidth during an earlier script.
Whichever solution is chosen, however, it would be highly advantageous to have a computer tool that enables delivery of information to be scheduled so as to optimize available bandwidth, while still enabling the playing of a multimedia script with minimal startup delay and no interruptions. Using this tool, the system designer would be able to orchestrate scripts linked to an optimum delivery schedule of the underlying media objects. When insufficient bandwidth is available for an instantaneous playback, the designer would further be able to make choices among the alternatives described above, such as delaying startup, redesigning script sequence and/or content, or preloading information, so as to make full and best use of available bandwidth. Further, in the context of UDTs according to the UDT Invention, the designer would be able to design an array of UDTs in which a particular information exchange could be scripted to be delivered at various levels of multimedia sophistication according to predetermined increments of available bandwidth.
Methods of identifying and quantifying available bandwidth in interconnecting links are known in the art. It will therefore be seen that given an array of UDTs as described above, each designed to deliver media objects on a schedule that fits within the constraints of an incrementally different bandwidth, it would then become possible, under the UDT Invention, to select a UDT to match the available bandwidth recognized in a particular interconnecting link.
Orchestrating the delivery of media objects within available bandwidth would further be advantageously enabled in a graphical environment on a general purpose multimedia-grade desktop computer. For example, scripting tools are known in the art where a designer may orchestrate the playback of a series of media objects by creating, adapting and executing a horizontal bar chart, much like a xe2x80x9cmusical score.xe2x80x9d There is a need in the art to attach this known orchestration and playback capability to corresponding functionality that shows how playback choices are affected by the constraints of delivery bandwidth. In this way, a designer may design, see, hear and manipulate scripts that also take delivery variables such as available bandwidth into account.
The disclosure of above-referenced U.S. application VISUAL AID FOR BANDWIDTH ALLOCATION IN MULTIMEDIA SCRIPTING TOOLS (the xe2x80x9cManual Inventionxe2x80x9d) (same effective filing data as the present application) describes an invention in which the system displays a xe2x80x9cdownload score,xe2x80x9d viewable in combination with orchestration and playback functionality. The download score enables a system designer to develop manually a xe2x80x9cdownload scriptxe2x80x9d of media objects that delivers information within a fixed bandwidth constraint (or xe2x80x9cpipexe2x80x9d) to meet the demand for data in the playback functionality. According to the Manual Invention, the system designer may adapt the shape of xe2x80x9cdownload iconsxe2x80x9d on the download store so as to fit within the pipe. When all the download icons fit within the pipe without creating an error condition, the system designer knows that the media objects that he or she has selected and scripted on the playback wore are deliverable within the bandwidth constraint that he or she selected on the download score. An disclosed in the Manual Invention, however, the designer may have adjust the playback script several times by a xe2x80x9ctrial and errorxe2x80x9d in order to create a playback score that is deliverable through the pipe. While the Manual Invention is useful for some applications, it is alternatively advantageous to be able to automate selected aspects of the creation of the download script according to predefined delivery rules, where the system automatically deploys download icons within the pipe according to the delivery rules and then adjusts their shape to fill up available bandwidth. In this way, the system designer could focus on developing aesthetically pleasing playback scripts deliverable within constraints of bandwidth without having to spend time manually adapting the shape of media objects to fit within a pipe on a display.
For the purposes of this disclosure, xe2x80x9cmedia objectxe2x80x9d or xe2x80x9cobjectxe2x80x9d is defined as an information structure, such as a file, which, when executed on a computer, plays the information to a user. A media object may be, for example, a sound clip, or a video clip, or a graphic. Media objects may come in any size or format suitable to be intelligible to a user through a variety of terminal devices.
In distinction to the foregoing definition of a media object, this disclosure also defines a xe2x80x9cmedia iconxe2x80x9d or xe2x80x9ciconxe2x80x9d as a representation of a media object on a terminal device, usually a computer screen.
The present invention is directed to a graphical representation of media object delivery in a limited bandwidth wherein the delivery of these objects may be optimized to make the best use of the available bandwidth. In a preferred embodiment, the invention presents three screens, or xe2x80x9cviewsxe2x80x9d to the system designer.
The designer advantageously first orchestrates a multimedia presentation by deploying media objects as a script on a xe2x80x9cplaybackxe2x80x9d score. The playback score allows the system designer to select and schedule media objects to be played back in overlapping sequence so as to make a predefined presentation. The playback score shows the media objects represented as playback icons relatively deployed on a time-indexed schedule, much like a musical score shows notes represented as recognizable marks on a stave. The designer further inputs additional parameters regarding the scripted media objects themselves. These additional parameters include media object size (when file size is not already known by the computer system), sequencing requirements with other objects, synchronicity requirements with other objects, and so on. Other media object parameters, such as time to start and time to stop, may also be recorded by the system from the designer""s placement of the playback icons on the playback score.
A second screen, or xe2x80x9cviewxe2x80x9d corresponding to the playback score is the runtime view. Advantageously viewable concurrently with the playback score by arranging multiple windows on the screen, the runtime view allows the system designer to see the executed presentation of the playback score in real time. In the runtime view, the system defaults to playing back the playback score at the same speed that the presentation is orchestrated on the playback score, from a start point to an end point in time as marked by the designer on the playback score. The designer may also speed up or slow down the runtime view""s execution as required to observe particular details. It will be understood, however, that the runtime view is a xe2x80x9cslave screenxe2x80x9d as far as orchestration of the playback score itself is concerned. That is, the designer uses the playback score as the tool to ordain the selection and substantive scheduling of media objects in making a presentation. The runtime view merely allows the designer to see the results of his or her work.
As the playback score and the corresponding runtime view are being developed, the present invention also develops a download score. This screen, or xe2x80x9cview,xe2x80x9d is a third view corresponding to the playback score, again also viewable by switching computer screens or manipulating multiple windows. In the present invention, the download score is a tool for the system designer to we the script embodied by the playback score (the xe2x80x9cPlayback Scriptxe2x80x9d) arranged by the system into a corresponding delivery schedule within predefined constraints of bandwidth (the xe2x80x9cDownload Scriptsxe2x80x9d). The system designer may thus create a Playback Script with a view to enabling the script to be played back intelligibly over a connection of limited bandwidth.
The system automatically or manually arranges download icons on the download score, each download icon corresponding to a playback icon on the playback score. The download score also displays a constraint of bandwidth previously ordained by the system designer, this constraint advantageously displayed as a separation of parallel lines, referred to herein as a xe2x80x9cpipexe2x80x9d. The system automatically or manually arranges download icons within the pipe according to predefined xe2x80x9cdelivery rulesxe2x80x9d, such a (1) not exceeding available bandwidth (i.e. not straying outside the pipe), (2) using up all available bandwidth within the pipe, and (3) ensuring that delivery of a particular media object on the Download Script keeps up with demand for it on the Playback Script.
As noted in a preferred embodiment, the present invention is enabled on software executable by a general purpose desktop computer. The computer is advantageously a multimedia-grade computer with capability to display alternately or concurrently the playback score, the runtime view and the download score, and further to play both visual and aural media objects in the runtime view according to the Playback Script. The software ideally runs under a well-known user-friendly operating system environment such as Microsoft(copyright) Windows.
As will be described in further detail below, in a preferred embodiment the system designer sees media objects as icons deployed on the playback score in an environment similar to other scripting tools known in the art. On the corresponding download score, however, the system designer sees media objects as icons distributed graphically within the xe2x80x9cpipexe2x80x9d of available bandwidth. The width of the pipe, of course, may be varied, scaled to represent corresponding variation in available bandwidth. The length of the pipe is indexed to represent time. The topographical size of an icon, when deployed in the pipe, thus represents the corresponding media object""s data size. The interrelated shapes and relative location of the icons as deployed within the xe2x80x9cpipexe2x80x9d thereby represent a delivery schedule of corresponding media objects within available bandwidth. As a result, the designer may develop Playback Scripts of media objects that are compatible with known constraints of bandwidth. If the designer takes an action on the playback score that causes a delivery rule error (e.g. pipe boundaries exceeded, delivery cannot keep up with demand), then the invention alerts the designer with an alarm. Advantageously, the computer highlights the icon(s) out of compliance with the rule on both playback and download scores. Different highlights may indicate different problems. The designer must then make adjustments (e.g. increase bandwidth, change media object content, move media objects, drop or substitute media objects) to compensate.
In a second embodiment, the system automatically or manually degrades the content of media objects (by, for example, resampling) according to preselected sequences and conditions ordained by the user. The corresponding reduction in data-richness of degraded media objects causes the Download Script to again be compatible with the Playback Script.
A primary application for the invention is thus the design of UDTs, so that bandwidth constraints may be taken into account when scripting media objects to be delivered to a user to enable an information transaction. The present invention is nonetheless not limited to this UDT application, and may be used whenever media object delivery needs to be optimability.
In UDT applications, it will be seen that an array of UDTs may be designed for each information transaction accommodating media object delivery not only in formats supported by the user""s terminal device, but also on a schedule supportable by the bandwidth of the interconnecting link. Advantageously, different UDTs support different ranges of bandwidth in which media objects are deliverable to support correspondingly information-rich scripts.
It is therefore a technical advantage of the present invention to optimize available bandwidth in delivering media objects.
It is a further technical advantage of the present invention to facilitate creation of an array of media object Download Scripts, each generally directed to delivering information in formats and/or media compatible with a particular receiving terminal device, each further specifically delivering selected media objects on a schedule deliverable within incremental ranges of available bandwidth.
It is a further object of the present invention to provide a designer with an interactive graphical tool with a playback score to design and adjust media object Playback Scripts within various bandwidth limitations. By also providing a companion download score on which the system automatically or manually arranges download icons to fit within a fixed pipe of bandwidth, the tool enables the designer to schedule the delivery of the media objects to meet the demand for data in the Playback Script.
It is a still further object of a preferred embodiment of the present invention to locate bandwidth automatically, so that the system designer can spend more time concentrating on aesthetic aspects of the multimedia presentation design. The automated aspect of a preferred embodiment of the invention allows the designer to make variations to the Playback Script based on aesthetic considerations, and then see/hear the results in the runtime view. Meanwhile, a preferred embodiment of the invention automatically allocates and reallocates bandwidth in the Download Script responsive to the designer""s changes to the Playback Script, and then alerts the designer if such changes have caused the Download Script now to violate one or more of the predefined delivery rules. The designer thus spends very little (if any) time adapting download icons on the download score to develop a Download Script (the system now doing this automatically), and the designer may instead spend more time developing aesthetically pleasing multimedia presentations using the Playback Script.
It is a still further object of the present invention to optionally enable automatic or manual degradation of media objects according to designer-selectable criteria in compiling a Download Script that is compatible with the Playback Script.
The foregoing has outlined rather broadly the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter which form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and the specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims.